Loading

×Sorry to interrupt
Uncaught ReferenceError: addCTracking is not defined
throws at https://kb.vmware.com/resource/KM_TealiumTrackingJS?v=0.5:100:1

Refresh

Cookie Preference Center

General Information on cookies

When you visit our website, we use cookies to ensure that we give you the best experience. This information does not usually directly identify you, but it can give you a more personalized web experience. Because we respect your right to privacy, you can choose not to allow some types of cookies by clicking on the different category headings to find out more and change your settings. However, blocking some types of cookies may impact your experience on the site and the services we are able to offer. Further information can be found in our

Cookie Policy.

Strictly Necessary

Strictly Necessary

Always Active

Strictly Necessary

Strictly necessary cookies are always enabled since they are essential for our website to function. They enable core functionality such as security, network management, and website accessibility. You can set your browser to block or alert you about these cookies, but this may affect how the website functions. For more information please visit www.aboutcookies.org or www.allaboutcookies.org.

Cookie Details‎

Performance

Performance

Performance

Performance cookies are used to analyze the user experience to improve our website by collecting and reporting information on how you use it. They allow us to know which pages are the most and least popular, see how visitors move around the site, optimize our website and make it easier to navigate.

Cookie Details‎

Please confirm your settings by refreshing the page.

Functional

Functional

Functional

Functional cookies help us keep track of your past browsing choices so we can improve usability and customize your experience. These cookies enable the website to remember your preferred settings, language preferences, location and other customizable elements such as font or text size. If you do not allow these cookies, then some or all of these services may not function properly.

Cookie Details‎

Advertising

Advertising

Advertising

Advertising cookies are used to send you relevant advertising and promotional information. They may be set through our site by third parties to build a profile of your interests and show you relevant advertisements on other sites. These cookies do not directly store personal information, but their function is based on uniquely identifying your browser and internet device.

Cookie Details‎

Advertising Cookies

Consent Leg.Interest

Select All Vendors

Select All Vendors

Select All Hosts

Select All

Clear Filters

Information storage and access

Apply

Confirm My Choices Reject All Allow All

My Subscriptions

Loading

Knowledge Base

VMware Performance Impact Statement for ‘L1 Terminal Fault - VMM’ (L1TF - VMM) mitigations: CVE-2018-3646 (55767)

Last Updated: 3/20/2024Categories: SecurityTotal Views: 91765Language:
Chinese (Simplified)JapaneseEnglish

13 like this article

Subscribe to get an email
newsletter with new updates

13 people found this helpful Please provide article feedback:

Purpose

Update 4/11/2019: VMware has published a white paper entitled Performance of vSphere 6.7 Scheduling Options which provides a detailed look into the performance differences between the original  ESXi Side-Channel-Aware Scheduler and the  ESXi Side-Channel-Aware Scheduler Version 2 which first shipped with ESXi 6.7u2.

This knowledge base article will be used discuss the performance impacts of security issues related to speculative execution described by CVE-2018-3646 (L1 Terminal Fault - VMM)  in modern-day processors as they apply to VMware.

Review the following Knowledge Base articles for a holistic view of VMware’s response to these issues:

KB55636: VMware Overview of Speculative-Execution security issues in Intel processors: CVE-2018-3646 (L1 Terminal Fault - VMM), CVE-2018-3620 (L1 Terminal Fault - OS), and CVE-2018-3615 (L1 Terminal Fault - SGX, SMM)

KB55806: VMware response to speculative-execution vulnerabilities in Intel processors: CVE-2018-3646 (“L1 Terminal Fault - VMM”)

Note: To aid in understanding the performance impact of Speculative Execution vulnerabilities to virtualization environments, VMware previously defined four mitigation categories in KB52245 and KB54951. Review these knowledge base articles for more detail if needed.

The Update History section of this article will be revised when there is a significant change to any of the related documentation. Click Subscribe to Article in the Actions box to be alerted when new information is added to this document and sign up at our Security-Announce mailing list to receive new and updated VMware Security Advisories.

Resolution

As discussed in KB55806, full mitigation of the “L1 Terminal Fault” requires mitigation of two currently known attack vectors:

Mitigation of the Sequential-context attack vector has minimal performance impact for enterprise applications. See the Performance Test Results section.

Mitigation of the Concurrent-context attack vector requires enabling the ESXi Side-Channel-Aware Scheduler, also referred to as the ESXi SCA Scheduler. Currently, this scheduler provides the Hyper-Threading-aware mitigation by scheduling on only one Hyper-Thread of a Hyper-Thread-enabled core. As described in more detail below, careful capacity planning is required prior to enabling the ESXi Side-Channel-Aware Scheduler as it could have a performance impact for enterprise applications.

Our tests showed that after enabling the ESXi Side-Channel-Aware Scheduler the maximum performance capacity of the host can diminish by as much as 30%, depending on the workloads, host utilization, and processors used within the host. It is important to note that this does not necessarily translate to a 30% reduction in application performance. On a host that is running below its maximum performance capacity, the enablement of the ESXi Side-Channel-Aware Scheduler might result in little or no loss of performance depending on how much spare capacity was available. A set of test results for common enterprise application workloads is included in the Performance Test Results section of this article.

Capacity Planning Considerations: Advanced CPU Settings

Important: Enabling the ESXi Side-Channel-Aware Scheduler may cause problems with some non-default advanced CPU performance settings, resulting in non-bootable VMs. Pay careful attention to the following:

If a VM has a total count of vCPUs greater than the total count of physical cores available on the host, the VM is considered too wide to run on the existing host. Prior to mitigation, this must be corrected by reconfiguring the VM to use fewer vCPUs or by moving the VM to a host with more cores.

Similarly, for a VM with manual NUMA affinity setting, the total count of vCPUs must not be greater than the count of physical cores available in the NUMA node.

For a VM with manual CPU affinity settings, the PCPU numbers may need to be altered prior to enabling the ESXi Side-Channel-Aware Scheduler as the former physical designations used may no longer be valid.

Enabling the ESXi Side-Channel-Aware Scheduler will effectively disable the numa.vcpu.preferHT setting at the host level. In some cases, a VM that uses the preferHT setting may need to be spread across more physical NUMA nodes on the host.

Capacity Planning Considerations: CPU Usage

Customers should conduct a thorough capacity planning exercise to estimate capacity requirements once ESXi Side-Channel-Aware Scheduler is enabled. VMware’s vSphere 6.7 Performance and Monitoring facility, and/or other system monitoring utilities can be used to gather information for this exercise. Note: Performance and Monitoring documentation for additional vSphere releases can be found on our VMware vSphere Docs page.

The severity of the impact caused by enabling the ESXi Side-Channel-Aware Scheduler largely depends on the CPU overcommit ratio on a given host, the host utilization, as well the application itself.
•    If the total count of vCPUs from VMs on a single host is less than the count of total physical cores available, that host should see little to no performance impact after enabling the ESXi Side-Channel-Aware Scheduler.
•    If a host is configured with CPU over commitment, the impact of enabling the ESXi Side-Channel-Aware Scheduler depends on the nature of the application. Consider the following two examples:

A virtual machine running an application that is purely throughput-sensitive, on a host that had as little as 30% spare capacity, may not experience any performance difference after the enablement of the ESXi Side-Channel-Aware Scheduler.

On the other hand, a virtual machine with a response-time sensitive application, under similar conditions, would likely see some increase in response time due to the high utilization after the mitigation enablement.

There is no single answer that covers every application. Customers need to conduct a capacity planning exercise and use the knowledge of their particular applications to evaluate whether or not the enablement of the ESXi Side-Channel-Aware Scheduler will cause a noticeable performance drop.

Capacity Planning Considerations: Resource Management Settings

The ESXi Side-Channel-Aware Scheduler is generally compatible with existing vSphere CPU resource management settings.

For example, enablement of the ESXi Side-Channel-Aware Scheduler will not impact existing CPU allocation specified by CPU reservations, limits, and shares on VMs or resource pools.

VMs with existing latency sensitivity setting of “high” are not impacted if the VMs are given full CPU reservation, as recommended.

Note: Whether or not the ESXi Side-Channel-Aware Scheduler is enabled, if the number of high-latency sensitive vCPUs >= (number of cores -1), then the “High” setting may not be honored.

The resource admission control and enforcement at host and cluster level will be honored.

Cluster-wide Capacity Planning Considerations

Rolling Upgrades: Due to reduced capacity at each host after mitigation, the cluster may not have the spare capacity for rolling upgrades. The capacity planning exercise should consider the effects of removing a host from the cluster and ensure the remaining hosts have sufficient capacity to absorb the application workload of the lost host.

VMware is providing the HT-Aware Mitigation Tool which may be helpful in analyzing your current environment. Refer to KB55806 for additional information.

Performance Test Results:

Patch applied: Sequential-Context attack vector mitigation:

Mitigation of the Sequential attack vector is achieved through a standard update to the product versions which will be listed in VMSA-2018-0020 when available. Below are the results of the performance impact observed in our test environments for enterprise class workloads.

Application Workload /
Guest OS

Performance Degradation after applying patch

Database OLTP / Windows

3%

Database OLTP / Linux

3%

Mixed Workload / Linux

1%

Java / Linux

<1%

VDI / Windows

3%

ESXi Side-Channel-Aware Scheduler enabled: Concurrent-Context Attack vector Mitigation:
Mitigation of the Concurrent attack vector requires enablement of a new option known as the ESXi Side-Channel-Aware Scheduler (ESXi SCA Scheduler) which will be included with the updates listed in VMSA-2018-0020. Performance impact of this mitigation depends on the remaining usable host CPU capacity prior to enabling the mitigation.

For example, consider the results below for an OLTP Database workload:

Application Workload /
Guest OS

% Host CPU Utilization before Enabling
the ESXi SCA Scheduler

Remaining usable Host CPU Capacity before enabling ESXi SCA Scheduler

Performance Degradation after enabling the ESXi SCA Scheduler

OLTP Database /
Linux

90%

0%

32%

77%

~15%

20%

62%

~30%

1%

Note: In this example, peak application throughput is reached at 90% host CPU utilization.

In the example above, the application reached its maximum performance with the host at 90% CPU utilization. We refer to this as having 0% remaining usable host CPU capacity. Since the mitigation was applied to an environment with 0% remaining usable host CPU capacity, the performance impact was greatest, 32%. However, the performance impact decreased significantly when the mitigation was applied to environments with larger amounts of remaining usable host CPU capacity.

While each application has its own characteristics, we saw a similar pattern across different types of applications. The more usable host CPU capacity available prior to mitigation, the less the performance impact was after mitigation.

Below are the results of the performance impact observed in our test environments for enterprise class workloads where the hosts were at or near maximum capacity prior to enabling the ESXi Side-Channel-Aware Scheduler.

Application Workload /
Guest OS

Performance degradation after enabling the ESXi Side-Channel-Aware Scheduler

Database OLTP / Windows

32%

Database OLTP / Linux (with vSAN)

32%

Mixed Workload / Linux

25%

Java / Linux

22%

VDI / Windows

30%

For further information refer to KB55806.

Detectable by VMware SkylineTM

Actions

Copy

link to clipboard
copied!

Print

Language:
Chinese (Simplified)JapaneseEnglish

Attachments

Additional Resources

KB
•
End Of General Availability of the Free vSphere Hypervisor (ESXi 7.x and 8.x) (2107518)

Result 1 of 1

Ask The Community

Get answers quickly from VMware experts in the community

Post Subject

CONTINUE IN COMMUNITIES

Related Products:

Datacenter

Related Versions:

Take Our Survey

Support Assistant